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ABSTRACT 


The v/ork reported herein was authorized under Contract flASl- 
13734 in 1974. This study was conducted under the direction of 
Patrick Gainer, Simulation and Human Factors Branch, Langley Re- 
search Center, NASA. 

This study was performed at Decision Science, Inc., San Diego, 
with Michael Mout and George Burgin acting as principal investiga- 
tors. 

This report provides a description of the Display Evaluation 
computer program, some results of this program and comparison of 
these results with a simple experiment. A detailed description of 
the experiment and data analysis is also included. 
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A COMPUTER PROGRAM 


FOR THE USE OF SENSITIVITY ANALYSIS. IN DISPLAY EVALUATION 

By Michael L. Mout and George H. Burgin 
Decision Science, Inc. 

SUMMARY 

This report describes a technique of computer evaluation of a 
visual display or set of visual displays. The operator of a vehicle 
has many cues by which to estimate the state of his vehicle. This 
method deals only with a subset of the visual cues available. These 
cues come from various displays which reflect changes in the state 
of the vehicle. 

These displays can reflect changes in any. one individual state 
variable or combinations of changes in the state variables. The 
theory upon which this method is based is that the operator of the 
vehicle estimates the vehicle's state variable values by responding 
to changes in the display. It is assumed that the operator has per- 
fect knowledge of the relationships between the display variables 
and the state variables. The only mistakes the operator makes are 
due to imperfection of his visual acuity in observing a given 
display. This acuity is assumed to be a function of where the 
observer is looking with respect to the center of the display. 

Mental errors such as control reversals cannot be measured by 
this method. 

The result of this method is a covariance matrix and a correla- 
tion matrix. These matrices reflect the amount of expected error 
in estimating each of the state variables and the amount of ambi- 
guity in the display between movement of two variables. These 
matrices allow the user to evaluate the various displays or sets of 
displays with respect to one another and also to evaluate various 
look points within a particular display. This evaluation may be 
used to indicate a general area that the pilot could scan in order 
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to limit the degree of error in estimation while minimizing the 
size of his scan path between Several instruments. 

The computer program for implementing this techniques has many 
options to allow the user as much freedom as possible in specifying 
the particular display or set of displays. The program at present 
is designed for four standard display types that might be seen in 
an aircraft. These are an eight-ball (for pitch, roll and yaw), 
an out- the-wi ndow display (such as a TV camera actually observing 
the real world), a circular dial type display, and a linear dial 
type display. There are many user specified options that may be 
used with these displays. 

Preliminary theoretical results confirm intuitive feelings 
about displays. For example, a large display has less error than 
a small display, look points at the center of the display show less 
error and less ambiguity between variables than look points away 
from the center, also, variables displayed separately have less 
error and are not confused with other variables. 

A simple experiment was conducted wherein a subject attempted 
to match up a movable cross with a fixed cross. This was done while 
fixating on various look points near the stationary cross. Results 
from this experiment were compared to a computer set-up of the same 
situation. This comparison tends to verify the technique for 
evaluating the worth of various look points within a display. 
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INTRODUCTION 


The controller of a vehicle, particularly a remote controller, 
must derive information about the state of that vehicle from visual 
cues. These cues usually come from displays that respond in fixed 
ways to the changes in the state variables of the vehicle. These 
displays could respond to only one state variable, several variables 
or to all state variables. In any case the functional relationship 
between the display movement and changes in the state variables can 
be derived analyti cal ly . 

The perfect controller of a vehicle with the perfect display 
or set of displays should be able to estimate the state variable 
values from the display variable values with no error. In other 
words the perfect controller sees exactly what the display vari- 
ables are and knows exactly what state variable contributed to the 
display variables (this assumes unique display variable values for 
each possible state variable value). Mathematically this situation 
can be stated as follows: The state variable/display variable 

function is one to one, the controller has absolute knowledge of 
this function, and the controller perceives the display without 
error. 

When a nonperfect controller with a nonperfect display makes 
an error in estimating the state variable values it can be due to 
any of three causes. One of these three causes, lack of absolute 
controller knowledge of the display state variable function cannot 
be directly measured and can change from controller to controller 
depending upon learning capability, experience, coordination, and 
so forth. The other two causes that could contribute to errors 
are a nonperfect display (display/state variable function is not 
one to one) and the lack of the controller to perceive the display 
variable values exactly. Both of these causes can be measured. 

The display/state variable function is either known exactly or can 
be approximated to some degree of accuracy. The ability of a 
controller with normal vision to perceive movement or distance is 
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called visual acuity and has been measured to a certain degree of 
accuracy for various visual angles. Using this acuity information 
and the relationship betv;een the display and state variables a 
measure can be derived of the accuracy that could be expected from 
a perfectly trained controller. , This measure would be dependent 
on some specific look point for each display. 

In 1973 a working paper (LWR1131) was prepared by Patrick 
A Gainer which laid the theoretical basis for this technique. This 
report describes a computer program which implements this technique 
for a wide variety of applications. This program is general enough 
to accept many kinds of input displays. It enables the user to 
evaluate various displays as he desires. The output from this pro- 
gram gives the user a measure of the relative value of the input 
display and of various look points within that display. Optionally 
this program includes maximum likelihood estimates of the state 
vector . 

Other tasks accomplished in connection with this computer pro- 
gram were to obtain experimental comparison data. These data are 
used to verify the worth of this technique in applying it to real 
world problems . 
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LIST OF SYMBOLS 


a 


1 J 


J 

X 

nv 

X* 




AB 


1 J 


Horizontal angle Clongitude) for location of the cen- 
ter of display segment i,j; 

1 = 1 , 2 , . i . n 
j = 1 , 2 , . . . m 

Vertical angle (latitude) for location of the center 
of display segment i,j 

Vector of state variables 

Number of state variables 

,C‘ denotes transpose) = 2 * ' * * ^nv ^ 

Change in horizontal angle of segment i , j due to 
change in X 

Change in vertical angle of segment i ,j due to change 



in X 


f(X;a.3) 

Function relating changes in X 

to a new 


angle for the segment centered 

at a, 3 

gCX;a,B) 

Function relating changes in X 

to a new 


angle for the segment centered 

at a, 3 

f,j(x) = 

j) 


g,j(x) = 

gCX;a,j.6,j) 


.1 

Influence matrix of partial derivatives 


^ij respect to all state 

vari abl es 

0 

Vector of the actual values of 

and 


1 J 
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A£ Vector of the actual values of Aa. . and A3- . for a 

' J * J 

given change in 21 

O' (a^ ^ , 3|^ , a-,2 , . . . , 3^^) 

A£' (Aa^-j, A3-|i, Aa^g 

£ An estimate of £ 

cov(£) Covariance matrix of the estimate of £ 

/N 

X Estimate of X derived from 0 


Acuity for segment i,j due to a given look point 
(See Graph 1 for the assumed acuity function 
used in this program.) 
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DESCRIPTION OF THE METHOD 

Basically this method assumes that the observer/controller's 
visual world can be thought of as a sphere of unit radius'. All of 
the visual cues that are available to the observer can be thought 
of as projected onto this sphere. This sphere (or a section of 
this sphere which contains information) can be divided into small 
rectangular segments (for example, 1° by 1®). Each of these seg- 
ments can possibly move whenever changes in the state variables 
occur. If very small segments are used then only vertical and 
horizontal movement need be considered. For small changes in the 
state variables the functional relationships between the segment 
movement and the state variables can be linearized. Thus the dis- 
play/state variable function will be linear. The display variables 
are the horizontal and vertical changes, of each segment. The 
linearized display/state variable function can be represented by 
the matrix I. This matrix contains the partial derivatives of the 
display variables with respect to the state variables. These der- 
ivatives are of the actual display/state variable function (analy- 
tically derived) at the values of the state variables being con- 
sidered. So in effect each element of the matrix I relates the 
relative change in a specific state variable to the horizontal or 
vertical displacement of a specific display segment. If we let 

and 

g,j = gCX;a,j.B,j) 

where g.. and f . . are functions relating changes in the state 

» J I xj 

vector, to change in the angles a.-,-; and g... 

Then the influence matrix I is given, by: 
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If the state vector X is multiplied by the matrix I, the- result is 
a vector ^ of observations of horizontal and vertical movement of 
each segment. 

0 = IX. 


Now if ^ changes by value, say AX, the corresponding obser- 
vation change A£ would be 




where 


Ao,, - 

3x^ 

^1 ^ 9x- 

AX2 + ... 

^ ^''''11 

Ae„ = 
• 

ag,, 

9Xi 

-9gi ^ 

'"''1 " ax” 

Ax 2 + ... 

. a g,„ 
3^v 

• 

• 

• 

^“nt« = 

nm 

3x.| 

Ax 1 

Ax 

9^nv 



9x^ 

Ax*| “t" 

^^m AX 



Now this change in ^ is observed by the viewer of this dis- 
play. This user wants to infer how this change in 0 was caused by 
a change in X. To do this the viewer estimates the new value of0_; 
call this estimation 2* Let 

✓s. 

where ^ is the error of estimation. The eventual goal is to de- 
rive an estimate of ^ from this estimate of Also, the distri- 
bution of the ^ estimate is needed in order to establish error 
limits on this estimation. To do this a multinormal distribution 
is assumed for £, a transformation of K is required to get a best 
estimate of (in a least squares sense) that has known distribu- 
tion parameters. 

From the above equation for 2 
5 = 0 - IX 

Let the multinormal Gaussian distribution of E have zero mean 
vector and a diagonal covariance matrix covC^)* The diagonal 
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elements are the variances of the error In estimating the horizon- 
tal and vertical position of the center of each segment. The 
standard deviation (square root of the variance) of this error is 
determined by the reciprocal of the observer's acuity in that part 
of his visual field. Both horizontal and vertical acuity are con- 
sidered to be the same for a given segment, so that if is the 

acuity for segment i,j then 



cov(O)"^^^ cov(£)“^^^ = cov(£)“^. 


Cov(£) ' is the matrix with the acuity values on the diagonal. 

If equation multiplied by cov(0)“^'*^^ , the resultant re- 

sidual vector cov(0)“^^^ (£-ir) weights each segment displacement 
by the acuity of that segment. The residuals ^ of this equation 
are now of the form 

i-(cov(0)-'/2^|. = (cov(0)-l/2Xo)-(cov(0)-'/2). (IX). 
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and £ has a multinormal distribution with zero mean and a covari- 
ance matrix equal to the identity matrix. This equation for £ can 
now be used to find a best estimate of X. Call this estimate %, 
and let it minimize the sums of squares of these weighted residuals, 

£ or 

min(£’£) = min{(cov(£)“^'''^6'' -cov(9)"^^^l£) ' ( cov (£) " ^ 

X l.~ 

-cov(0)"^/^IX)} = miri{(0 - IX)' cov(6)"^ (6-IX) }. 

1 - - 

Solving the resulting normal equations gives 

X = [rcov(6)"^ I] ^ r cov(o)"^ o. 

Now if £ is of the form £ = a£ then 

cov(£) = A cov(6) A' (ref.'l, p. 79). 

Here 

A = [I'cov(£)"^ I] I' cov(£)”^ 

then 

cov(£) = [I' cov(5)~^ I] ,1' cov(£)"^ cov(6) 

- -1 

(r cov(£) ) [I' cov(O) I] 

/V . 

and since cov(^) is a symmetric matrix this reduces to 
cov(£) = [r cov(6)"^ I] 

This is basically the method that the computer program is 
written to calculate. 
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THE COMPUTER PROGRAM 


This program has evolved through several stages but will be 
described here in its final form. The program is basically 
written for a vehicle that is moving in space with possible 
movement in six degrees of motion (three Euler angles and the three 
translation axes). This movement results in changes of segments 
on a sphere of unit radius. These changes are dependent on 
specified functional relationships between the sphere segments 
and vehicle movement. 

In order to better understand the workings of this program a 
short example will be considered. Let the controller be seated 
one meter from his display panel , (see Figure 1). At this distance 
1.75 cm. subtends approximately 1 degree of arc. On this panel are 
two displays. One display shows range x on a linear scale. This 
display is approximately 5 cm. high and 23 cm. wide, with a moving 
pointer to show relative range. Another display which is 60 cm. 
wide by 40 cm. high is a TV camera showing the outside view of the 
vehicle. The center of the panel is located straight ahead of the 
controller. The center of the linear display is located 33 cm. to 
the left of the center and 16.5 cm. down. The center of the TV 
display is located 24 cm. to the left and 33 cm. above this center 
line. 

The computer program will generate measures of how well the 
controller can estimate change in these displays from some nominal 
state variable conditions. These measures are the amount of ex- 
pected error and the confusion measure (correlation) between state 
variabl es . 

This example is shown in more detail further on in this chap- 
ter along with the corresponding computer set-up and output. 

The first task performed by the computer program is to set up 
the display or displays (two or more displays may be considered 
simultaneously). This set-up is for the display size, location. 


12 



and segmentation. Each, display has a.speclfied function that re- 
lates segment movement (vertical and horizontal) in that display 
to the state vector changes. This movement may be in response to 
only one. state variable, as in an altimeter, or to all state vari- 
ables, as in an out-the-wi ndow view or a TV camera view. 

When this set-up has been completed, the program then calcu- 
lates the elements of the influence matrix. These elements relate 
the changes of each state variable to changes in the vertical and 
horizontal position of each segment. The partial derivatives here 
are found by numerical differentiation. 

Now the program is ready to read in a series of look points 
for each display in this set-up. These viewpoints can be anywhere 
on the sphere but are usually located near the display center being 
referenced. Once the look points for each display is read in the 
visual acuity values can then be calculated for each segment. The 
values are a measure of how well the observer can estimate either 
displacement (position) or rate-of-movement of that particular seg- 
ment while fixating on the given look point. These acuity values 
are used to calculate the covariance matrix of the observation vec- 
tor which, in turn, is used in calculation of the covariance matrix 
of the state vector. The covariance matrix of the state vector 
along with the correlation matrix are then output for the user's 
reference. 

The program optionally generates a series of perturbations in 
the state vector and calculates the maximum .1 i kel i hood estimate 
(MLE) of each of these perturbations. This . procedure can be re- 
peated for the various look points in this display set-up. 

Various options are available allowing the user to specify 
the display size, segment size, functional relationships, size of 
state vector perturbations, step-size in the numerical differen- 
tiation, overall acuity weights for a particular display (this to 
reflect possible degradation of a display either visually or mech- 
anically) weights for individual look points in a display set (to 
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reflect, memory retention or lack of retentionl, and individual seg- 
ment weights within a display (to reflect the amount of useful in- 
formation or cues in that segment). Furthermore, this program is 
set up to allow the user to program his own special display/state 
variable functions with a minimum of changes in the total program. 
New acuity functions may also be introduced. 

An Example 

Tables T through. 5 show the set-up and results of the example 
discussed at the beginning of this section. 

The first display is a linear display of variable 4 (range or 
"X"). This display moves horizontally as X changes. This display 
is 14° by 3° with segments size 2° by 1°. The first display is 
located to the left and down from the observer's center. The second 
display is a real world picture. It is located to the left and up. 
This display is 35° by 25° in size with segment size of 5° by 5°. 
This display shows information from a camera or periscope pointing 
20° to the right of the center of the vehicle. The picture being 
portrayed is degraded (this to model a TV with too few scan lines, 
a dirty camera lens, clouds or haze, etc.), and has an acuity 
weight of .6827. Tables 3 through 5 show the results from the 
first set of look angles for this display set. Table 3 shows the 
look angle and the final weight for each display. The final weight 
for look system (display) two now is .6144. This is less than the 
original acuity weight of .6827. The reason for this is that the 
final weight is the product of the acuity and the memory retention 
weight. In this case display two has a .9 memory weight given a 
final display weight of (.6827)(.9) = .6144. 

This display set has the option to read-in the segment weights. 
In display 1 the weighted segments are the second column of seg- 
ments. This is to indicate a pointer on the linear scale. Display 
2 has weighted segments in the middle of the display. This indi- 
cates that the center of this display contains more information 
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than the peripheral areas. The resulting correlation matrix from 
this analysis, shows that variable 4 C"X" or range) has a much lower 
standard deviation and has no strong correlation with any of the 
other variables. This is because the "X" variable is the only 
variable on the linear display (display 1) 

In these tables of results there are eigenvector analysis re- 
sults. These matrices can be used to find an orthogonal space to 
generate new variables that are independent. These matrices have 
no immediate use in the present analysis but their meaning is being 
investigated for future use. 
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EXPERIMENTAL VERIFICATION 


A search was made for experimental data that could be used to 
verify this particular display evaluation method. In the past, 
displays have been evaluated by comparing performance between two 
or more displays. This performance has usually been measured on 
some analog type device giving some total measure at the end of a 
run. The data needed for experimental comparison to this method 
had to be in some form of multivariate observations from which a 
covariance matrix could be calculated. Also, information on the 
look angle for these observations had to be available. No experi- 
mental data could be located which satisfied these criteria. Due 
to this, a simple experiment was designed and conducted at the 
Decision Science, Inc. facility. A full description of this ex- 
periment and the results follows. A summary of these results will 
be repeated here. 


Summary of Results 

The theoretical covariance matrices were calculated for two 
different weighting schemes on four experimental conditions. The 
first weighting scheme was a gross approximation of the actual 
display and resulted in theoretical variances which were much lower 
than the sample variances. The second weighting scheme was an 
attempt to more precisely reflect the actual display. This second 
weighting scheme resulted in much larger theoretical variance 
values. The values were in many cases not significantly different 
from the sample variances. Under both weighting schemes most of 
the sample correlations were not significantly different from the 
theoreti cal . 
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Experimental Conclusions 

The primary conclusion to be drawn from these results is that 
the basic computer model of the experiment can drastically affect 
the results. When using the proper setup, the algorithm seems to 
be a valid model of human performance. There are many consistencies 
noted between the sample covariance matrices and the theoretical 
matrices. The presence of the consistencies give strong indication 
that further experimental data would be valuable for a more in- 
depth verification of this method. 

The Experiment 

A simple experiment was conducted as follov/s: 

Display .- An overhead projector showed a cross to simulate the 
pitch, roll and yaw (e, (j> , and ip) of some vehicle (see Figures 2A, 
2B, 2C). 

Look Point .- The observer was instructed to look at some point 
(a dot on the display, see Figures 2A, 2B, and 2C) in this display. 

Estimation of State Variables .- The subject then placed a 
moveable crosshairs in line with the displayed crosshairs while 
continuously looking at the dot. If the subjects look point changed 
while the subject attempted to match up the crosshairs, the data 
were discarded. 

Measurement .- After the observer estimated the state variable, 
measurements of error of these estimates were taken (see Figures 
2B and 2C). These values were converted to degrees and minutes of 
arc. The errors were then used to get an estimate of the covari- 
ance matrix which v/as compared to the original matrix produced by 
the method being tested. 
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Experimental Results and Comparison 
to Theoretical Data 


Subjects and Data CoTTection .- One subject was used on this 
experiment with two different display configurations and two look 
points for each display. There were 12 observations taken under 
each of these four conditions. A different subject was tested 
under the first condition CH observations) in order to verify that 
there did not exist any great subject variations. 

In order to derive the theoretical results each experimental 
condition was converted to a set-up compatible with the computer 
program. This basically involved transforming the data on look 
points, display size, and so forth into degrees of arc. Also, the 
actual display had to be put in a segmented form. 

The four experimental conditions and the respective computer 
set-ups are shown in Figure 3A, 3B, 3C, and 3D. The distance from 
the subject to the displayed view was 165.0 cm. This distance re- 
sulted in a display of 17° x 17° with segments of 1° x 1°^. The 
various weights for the segments and look point locations are shown 
in Figures 3B and 3D. The output from the computer program shown 
in Tables 6 through 14 gives the set-up in numerical form and also 
the resultant theoretical distribution parameters. 

Data Preparation . - A computer program was written to convert 
the observation data from distance to minutes of arc then calculate 
the corresponding covariance matrices for each of the experimental 
conditions. The conversion from distance measures to minutes of 
arc was accomplished by the formula: 

min bf ..arc = 60 X 'TAN“|(^./d) 

where Z = length to be converted 

and d = distance from the viewer to the picture 

The distance to the picture in this case was 165 centimeters (cm.) 
For example, in Figure 2A the cross length is 42 cm. which converts 
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to 

857 min of arc = 60 x TAN“U42/165) 

or approximately 14° as shown in Figure 2B. 

The program also calculates the rotation angle, (j>, from the 
change in slope of the estimated crosshairs with respect to the 
displayed crosshairs; this is done over a 10 cm. length. Here two 
measures 10 cm. apart, A and B (see figure 2C),.of the vertical' 
distance from the display horizontal axis to the estimated horizon- 
tal axis are taken. The angle (p is then found fay the following 
formula 

(1) = 60. X TAN"^ ((A-B)/10. ). 

The results of this program are shown in Tables 15 through 19. 
Tables 15 and 16 are for the same experimental conditions with two 
different subjects. Tables 17, 18 and 19 show results from the 
first subject under three different experimental conditions (see 
Figures 3A, 3B, 3C, and 3D for experimental conditions). These 
tables also show the experimental data used to derive each of the 
matrices. 

Analysis and Discussion 

The results comparing the two different subjects using display 
A, look point 1, show no significant difference in the covariance 
matrices. Using Box's test for equality of covariance matrices 
(Ref. 1, p. 158), a chi-squared value of 7.82 (six degrees of free- 
dom) is calculated. This value has a corresponding p-value of .25 
Cgreater than .05) under the hypothesis of equal covariance ma- 
trices. The mean vector values are easily within the one mean 
standard deviation of zero so that the conclusion of no significant 
subject difference is made. Due to this result the remaining tests 
were run with only one subject, this being due to time, personnel, 
and cost restraints. (Tables 22 and 23 show the data from these 
two subjects.) 
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In comparing the theoretical results with the experimental 
results several important items should be noted. Of prime impor- 
tance are the restrictions that the computer program places on the 
model. The true condition cannot be reflected exactly, particular- 
ly in the segmentation and weights. For simplicity's sake, a seg- 
ment was given a weight of one if a line goes through the. middle 
of it and a weight of 1/2 if the line crosses a corner (see Figures 
3B and 3D). It is possible that smaller segment si zes: ( requi ri ng 
programmi ng changes ) or more carefully calculated weights would 
give results closer to the real world experiment. 

Table 6 shows the display set-up. Tables 7 through 14 have 
the preliminary theoretical results. Tables 15 through 19 show 
the experimental data and results. - 

In general, the experimental results show much higher vari- 
ances as opposed to the theoretical results (see Table 28). This 
seemed to be due in a large part to the grossness of the segment 
weights for the different displays. Because of this the segment 
weights were calculated to more realistically reflect the actual 
display. This was done by carefully drawing the real display in 
the segmented sections. The weights were then calculated for each 
segment as the percentage of that segment that was filled. This 
resulted in the segment weights shown in Figures 4A and 4B. Here 
the width of the display lines was taken to be 3.2 mm. or .11 de- 
gree of arc. Since the segments were 1° x 1° a segment with two 
lines through it (see the center segment of Figure 4B), each 1® 
long, would have a .22 weight. 

The results of this weighting scheme are shown in Tables 20 
through. 27. Here the theoretical variances are much larger than 
with the previous weights and in many cases are not significantly 
different from the experimental results (see Table 28, graph 2, 
and graph 3 for these comparisons). This new weighting scheme 
changes the correlations somewhat, but not drastically. Several 
of the sample correlations were significantly different from the 
theoretical, but in most cases the differences were small (see 
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Table 29 for comparison and statistics on correlation). 

The comparisons between look points in a given display and 
within the covariance matrices for a given look point are for both 
the theoretical and experimental results. The experimental data 
shows larger variances on the (|) variable than 0 and ip. In general, 
the variance of the experimental data, decreases as the look angle 
decreases. Both of these results are seen in the theoretical par- 
ameters . 

Overall, the results of this simple experiment seem to justify 
the use of this algorithm for evaluating the comparative worth of 
different displays and different look points within a display. 
Further experimentation with more observations per look point, 
more variety of look points, displays of varying acuity and use of 
more subjects would be justified for a more absolute verification 
of the evaluation techniques. 
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CONCLUSIONS 


The basic conclusion to be made from this study is that the 
technique described herein is a valid technique for evaluating 
displays. The computer program generates results on simple dis- 
plays that are agreeable with intuition. The computer program is 
presently flexible enough to allow the user to investigate a wide 
variety of combinations of displays. These displays may be ana- 
lyzed as being viewed simultaneously or sequentially. The program 
can be modified to include special types of displays not presently 
programmed . 

Overall, the computer program of this technique can be util- 
ized to evaluate the relative worth of any number of theoretical 
displays without actually constructing these displays. Using this 
program for calculating the absolute worth of a display would re- 
quire more careful verification of the technique and certainly a 
precise modeling method of the actual display. 

The basic problem in applying this technique to human perfor- 
mance is the nonaccountability of mental errors. This would re- 
quire the actual modeling of the mental reaction pattern of the 
subject in question. The best that can be expected of this tech- 
nique is to give the best limitations of the performance that could 
be expected from the perfect human operator. 
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RECOMMENDATIONS FOR FUTURE WORK 


Primarily, the item most lacking in reference to this techni- 
que is in-depth experimental verification. This would require an 
experiment explicitly designed to generate data compatible with 
this technique. The ideal, situation for this type of experiment 
would require an eye-tracking device and a computer interface with 
some sort of display/control equipment. 

Further extensions of the computer program could involve the 
inclusion of more display/state variable functions, these to pos- 
sibly reflect relationships other than vehicle movement. Other 
possible additions could be that of deriving and generating further 
evaluation measures based upon the eigenvector analysis now in- 
cluded in the program. 

The most promising extension of this technique is to expand 
the program to include displays that are changing with time. This 
would include the expansion of the technique to update the covar- 
iance matrix and state. vector estimation with each change in 
look point and display change. The results of this extension to 
the algorithm could be used to find the limitations of a pilot in 
a nonstatic situation. These limitations would be expressed in 
the amount of error that could be expected due to certain aircraft 
maneuvers and scanning patterns of the displays. 

Decision Science, Inc. 

San Diego, California, July 6, 1976 
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TABLE 2.- OUTPUT SHOWING DISPLAY PARAMETERS 
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TABLE 3 

OUTPUT SHOWING THE LOOK POINTS AND SEGMENT WEIGHTS 



INVeRS OF COVARIANCE MATRIX 
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TABLE 5 

FINAL OUTPUT FOR THE SPECIFIFD LOOK POINTS 
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RUM WITH r-XPER I mental. jATA GROSS HEIGHTS. 


NUMBER OF STATE VARIABLES 3 


ESTIMATION IS fur POSITION OF STATE VARIABLES 


INPUT OATA IS IN DEGREES 


WEIGHTS MILL BE READ IN 
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n DEGREES from VERTICAL. 

OR DISPLAY 

Ts 0 degrees horizontal 


*** FUNCTIONAL FORM i FOR 

THIS system 



form 1 is a real WORLD SPHERE, 
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rOF?M 3 IS A linear 

DISPLAY. 



,»****■ L 0 n K System *** 

L at I DUD INAL SEGMENT LENGTH 

1 ********* 

1 . 0 0 G 0 0 


LONGlTuniNAl, segment LENGTH 
MAXIMUM I.ATIOUDINAL ANGLE 


1 . OQOOO 

8, ogcoo 


maximum LOl'ic I tuu I nal angle 

LIMIT FOR CHANGts 


8.0 0 0 0 0 
1.00000 


AMOUNT OF change 

EPS I lot. for difh erent I a i ion 


i.ooooo 

. 0200.0 

DISPLAY I'AilAMETERS 

0 

0 

0 

number of bi-EMENTS 


269 



TABLE 6.- SET-UP OF DISPLAY PARAMETERS 
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LATITUDINAL LOOK ANOLE LAMBDA 3ETA 19.50000 



TABLE 7 

DISPLAY A - FIRST LOOK POINT PARAMETERS 
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TABLE 8 


DISPLAY A - FIRST LOOK POINT RESULTS 
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TABLE 9 

DISPLAY A - SECOND LOOK POINT PARAMETERS 















IMVERS of covariance matrix 


.206^^81 -. 000479 , 009343 

" . 000479 .203609 . 006469 

. 009343 _ •j'506_^69 ^00l7Q6 


E I G E F-l vector analysis 

COVARIANCE MATRIX 

6.756920 1.339699 - 4 2"^^ « 2 9 5 ‘5 


1 .... 339Q99 5 , 787957 -•29,2^1893 


-42.082955 -29.281893 927.625346 


E3 0BNVECX ns- ^_ : 

." 9 si 2 91 ’ ■;i§22AQ -■;o'45534 


-:..l.93/6fl ^9.,a43ia^ .031684 


.038557 .039882 

,996460 

ru APAPTCP T T f- WnnTR 

4.836VQ9 4.859603 

930.473711 


determinant of UiE COVARIANCE MATRIX = 

21871.2139435 


correlations anu variances 

IN minutes of arc. 


2.599408 

.214225 

-.531552 

.214225 

2.405817 

-.399623 


-.531552 

- .399623 

30.456942 


TABLE 10 

DISPLAY A - SECOND LOOK POINT RESULTS 



LATlTUDll'Mt LOOK ANGLE LaMBOA BETA -lO.'^dOOO 

LONnlTuniMAL LOOK ANGLE ! AMBUA Ai PHA - 9 . 09 nnn 

DEGRADATIO-'I FACIqR -r(MEMoRY AND DISPLAY) l.OOOQO 



TABLE n 

DISPLAY B - THIRD LOOK POINT PARAMETERS 
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A .N A i. Y S I S 


C0VARIA’'JCE MATRIX 
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-.079470 

^704031 _ 

-3.156897 


4. 011!393 

-3,156897 

159.332139 

EIGENVECTORS 
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1 57U73 
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^M99D8 


-.028115 
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.999432 

CHARACTERISTIC ROOTS 

.838425 • 

.841220 

159 . 496555 




determinant of !|HE COVARia 

MCE MATRIX = 

112.4929553 


CORRELATIONS ANU VARIANCpS 

IN mtmotes of arc. 

. 969852 

-.086206 

.327789 

. n 86^ n 6 

.95080 6 

- .26.3037 

t' 

.327789 

-.263037 

12.622683 


TABLE 12 

DISPLAY B - THIRD LOOK POINT RESULTS 
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DISPLAY B - FOURTH LOOK POINT PARAMETERS 



INVERS Of- Co Variance matrix 


3 . 1?2X23 

. n 0 0 u 0 n 

.000050 
3 . 115520 

.112465 
- . 032044 

. 112465 

. 032044 

. .niQR4«5 


E I G E t-! V [• C T 0 R A 

('! A I,, Y s I 

s 

C 0 V A R I A MCE MATR . i 

.404205 

- . 023964 

-2.329414 

> .0 239^4 

. 327flnG 

■ 66R1 26 


-2.3294x4 

.665126 

54.656021 

E I GFNVE'^TDR^ 


. 993935 

.106603 

- . 0 36176 

- . 1 TiAvS 7 6 

- 99427? 

- 1.1 n32Q 


c: 

- . 006416 

.999292 


CHA R A CTFR I STIC -H n Q.L5 

.339^65 .3209^12 54.757224 


determinant of Ire covariance matrix = 6.6478464 


I C0RRELATI0i-i5 aNU VARIANCES 

IN MINUTES OF ARC, 


.635/71 

- .0 65835 

- .455625 

- . 0 6 b 3 5 

.572542 

.144464 


-.45.5bp5 

.144464 

8.041519 


TABLE 14 

DISPLAY B-- FOURTH LOOK POINT RESULTS 
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EXPERIHENTal SET*UP "“DIsTANCI 165.10CM, 

LOOK point location x 8 OCMt Y f 59.4ACM. 

mCONAX cROSS" straight UP LOOK PQINf, DISP A, LOOK PT, 1 S08J 
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experimental SET*UP Dl^tlNCe TO DjsPUY = 169,10CM. 

LOOK POINT LOCATION _x s "33.65CM» Y g 48,QtCM, 

diagonal- diSPLAYToOK pT. left upper, “dISR a, look PT, SUBJ 
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experimental set-up distance to display s 165.10CM, 

I^OK POINT LOCATION xj lg,19CM, Y » 26.04CM. 

STRAIGHT CROSS LOOK PoJNT UPPER RIGHT, OISP B. POOK PT 4. SUBJ 
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DISPLAY A - LOOK POINT 2 - FINE WEIGHTS 
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PARAMETERS - DISPLAY B - LOOK POINT 4 - FINE WEIGHTS 
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Correlations for Different Weighting Modes 
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